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Abstract — Erasure correcting codes are widely used to ensure 
data persistence in distributed storage systems. This paper 
addresses the repair of such codes in the presence of simultaneous 
failures. It is crucial to maintain the required redundancy over 
time to prevent permanent data losses. We go beyond existing 
work (i.e., regenerating codes by Dimakis et al.) and propose 
coordinated regenerating codes allowing devices to coordinate 
during simultaneous repairs thus reducing the costs further. We 
provide closed form expressions of the communication costs of 
our new codes depending on the number of live devices and the 
number of devices being repaired. We prove that deliberately 
delaying repairs does not bring additional gains in itself. This 
means that regenerating codes are optimal as long as each 
failure can be repaired before a second one occurs. Yet, when 
multiple failures are detected simultaneously, we prove that our 
coordinated regenerating codes are optimal and outperform un- 
coordinated repairs (with respect to communication and storage 
costs). Finally, we define adaptive regenerating codes that self- 
adapt to the system state and prove they are optimal. 

Keywords — erasure correcting codes, regenerating codes, dis- 
tributed storage, repair, multiple failures 

I. Introduction 

Over the last decade, digital information to be stored, be it 
scientific data, photos, videos, etc, has grown exponentially. 
Meanwhile, the widespread access to the Internet has changed 
behaviors: users now expect reliable storage and seamless 
access to their data. The combination of these factors dramat- 
ically increases the demand for large-scale distributed storage 
systems. Such systems are used as back-ends by cloud service 
providers or as a basis for P2P systems to provide users with 
storage, backup or sharing capabilities. This is traditionally 
achieved by aggregating numerous physical devices to provide 
large and resilient storage pl-p). In such systems, which 
are prone to disk and network failures, redundancy is the 
natural solution to prevent permanent data losses. However, as 
failures occur, the level of redundancy decreases, potentially 
jeopardizing the ability to recover the original data. This 
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requires the storage system to self-repair to go back to its 
healthy state (i.e., keep redundancy above a minimum level). 

Repairing redundancy is of paramount importance for the 
design and implementation of distributed storage systems. A 
self-healing mechanism is usually composed of three phases. 
First, the system must self-monitor to detect failures. Second, 
the system must trigger a repair on a set of spare devices. 
Finally, the system must regenerate the lost redundancy from 
the remaining one. In this paper, we focus on this last 
phase. Redundancy in storage systems has been extensively 
implemented using erasure correcting codes |6|-|8| because 
they enable tolerance to failures at a low storage overhead. In 
this context, the repair used to induce a large communication 
overhead, as it required to download and decode the whole file. 
Yet, Dimakis et al. recently showed j9j, fit)) that the repair 
cost can be significantly reduced by avoiding decoding in the 
so-called regenerating codes. 

In this paper, we go beyond these works by considering 
simultaneous repairs in regenerating-like codes. We propose 
coordinated regenerating codes allowing devices to leverage 
simultaneous repairs: each of the t devices being repaired 
contacts d live (i.e., non-failed) devices and then coordinates 
with the t — 1 others. Our contribution is threefold: 

• As deliberately delaying repairs in erasure correcting 
codes leads to savings |5), fTT) , fl2| , it is natural to 
wonder if the same additional savings can be expected 
when delaying repairs for regenerating codes. By defining 
coordinated regenerating codes, we prove that, when 
relying on regenerating-like codes (MSR or MBR) |10|, 
deliberately delaying repairs (so that t > 1) cannot lead 
to further savings. 

• Yet in practical systems, it might be difficult to detect 
every single failure and fix it before a second one occurs 
(i.e., ensure t = 1). We establish the optimal quantities 
of information to be transferred when t devices must 
be repaired simultaneously from d live devices. Our 
coordinated regenerating codes consistently outperform 
existing approaches. 

• In addition, most practical systems are highly dynamic. 
Therefore, assuming that t and d remain constant across 
repairs is unrealistic. To address this issue, we define 
adaptive regenerating codes achieving optimal repairs 
according to t and d. Under a constant system size d + 1, 
their performance does not depend on t (i.e., the cost of 
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(a) Erasure correcting codes (immediate repair) (b) Erasure correcting codes (delayed repair) (c) Regenerating codes 

Figure 1. Repairing failures with codes. In a n device network, failed devices are replaced by new ones. The new devices get a given amount of data from 
live devices to repair the redundancy. In our examples, fe = 3, d = 4, B = 1, a = 1, and /3 = 1/2. Some example of gains for k = 32 are given in Table III 



a repair does not increase if the number of failed devices 
to repair increases). 

Previous approaches are either known for not supporting 
simultaneous coordinated repairs [10] or known for assuming 
that repairing implies decoding which requires to download the 
whole file (5)-||7jj fTT| , fl2| . Hence, we define coordinated 
regenerating codes that fill the gap between the two aforemen- 
tioned approaches by achieving simultaneous repairs without 
decoding (Figure [4). Furthermore, as existing regenerating 
codes assume a static set of parameters, we define adaptive 
regenerating codes that can self-adapt. Two recent works have 
been interested in these same problems. MCR Codes [ 1 3 1 
define MSR-like codes that support multiple repair at once 
and MFR [ 14 1 codes show that MSR codes can be turned into 
adaptive codes. Yet, MCR Codes only consider the Minimum 
Storage point and assume that all transfers are equal without 
proving it (i.e., f3 — (3'); and MFR fl4) codes are not 
optimal when repairing more than one failure. Finally, since, 
for erasure correcting codes, simultaneous repairs reduce the 
communication overhead, we study the impact of deliberately 
delaying repairs with regenerating-like codes. 

The paper is organized as follows. Section \U\ describes the 
background on codes. Section III presents our coordinated 
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regenerating codes. In this section, we prove the optimality 
of our codes, we derive closed-form expressions for specific 
subsets of codes, and we show that deliberately delaying 
repairs does not reduce the costs further as long as each 
failure can be fixed before a second one occurs. In Section [Tv| 
we propose adaptive regenerating codes that self-adapt to 
the dynamic of the system and we prove their optimality. 
Section [V] briefly reviews some related work. Section [VT] 
concludes. 

II. Background 

We consider a n device system storing a file of A4 bits split 
in k blocks of size B = A4 jk. To cope with device failures, 
blocks are stored with some redundancy so that a single failure 
cannot lead to permanent data losses. We consider code-based 
approaches to redundancy since they have been proven to be 
more efficient than replication with respect to both storage 
and repair costs (6). We focus on self-healing systems (i.e., 
systems that automatically repair themselves upon failure). 
Distributed storage systems are required to be self-healing so 
as to ensure that the system does not gradually lose its ability 
to recover the initial file. Self-healing systems are equipped 



and triggers repairs 1 15 1 on new spare devices. To repair, the 
new spare devices regenerate the lost redundancy from data 
downloaded from live devices. The repair is constrained by 
the communication costs (16). In the rest of this section, we 
describe the main code-based approaches for generating and 
repairing redundancy. 

A. Erasure correcting codes (immediate/eager repairs) 

Erasure correcting codes have been widely used to provide 
redundancy in distributed storage systems [6|-|8|. Devices 
store n encoded blocks of size B, which are computed from the 
k original blocks. As erasure correcting codes are optimal with 
respect to recovery (i.e., they allow recovering the k original 
blocks from any subset of k encoded blocks), storing encoded 
blocks at n = k + / devices is sufficient to tolerate / failures. 
This approach is efficient with respect to storage. Yet, repairing 
a lost encoded block is very expensive since the devices must 
fully decode the initial file. Hence, repairing a single lost block 
implies downloading k encoded blocks as shown on Figure la 



B. Erasure correcting codes (delayed/lazy repairs) 

A first approach to limit the repair cost of erasure correcting 
codes is to delay repairs and factor downloading costs (5), 
(TTJ, p2| . When a device has downloaded k blocks, it can 
produce as many new encoded blocks as wanted without 
any additional cost. Therefore, instead of performing a repair 
upon every single failure (Figure [2a]), repairs are deliberately 



delayed until t (threshold) failures are detected (Figure 2b i 



This repair strategy is depicted on Figure lb One of the 
new devices downloads k blocks, regenerates t blocks and 
dispatches them to the t — 1 other spare new devices. 
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Figure 2. Delaying repairs allows performing multiple repairs at once. 
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C. Network coding and regenerating codes 

A second approach to increase the efficiency of repairs 
relies on network coding. Network coding differs from erasure 
correcting codes as it allows devices to generate new blocks 
with only partial knowledge (i.e., with less than M. bits). 
Network coding was initially applied to multicast, for which 
it has been proved that linear codes achieve the maxfiow in 
a communication graph fT7|-|fT9|. Network coding has latter 
been applied to distributed storage and data persistence [20 1— 
1 23 1 . A key contribution in this area are regenerating codes |9|, 
1 10 1 introduced by Dimakis et al. They infer the minimum 
amount of information to be transferred to repair lost redun- 
dancy. 

The idea behind regenerating codes |9) is that, when com- 
pared to erasure correcting codes, more devices are contacted 
upon repair but much less data is downloaded from them thus 
offering low repair cost for each single failure. Similarly to 
erasure correcting codes, n encoded blocks of size a bits 
are computed from the k original blocks. Each device stores 
a « 4r bits. During a repair, the new device contacts d > k 
other devices to get j3 <C M/k bits from each and stores a 
bits as shown on Figure [Tc| 




MSB 



EC- 



Repair cost (bandwidth) (7) 

Figure 3. Regenerating codes (MSR or MBR) offer improved performances 
when compared to erasure correcting codes (EC) 



Regenerating codes achieve an optimal trade-off between 
the storage a and the repair cost 7 = d/3. The graph on 
Figure [3] depicts the performance of the optimal regenerating 
codes. Points (a, 7) above the curve correspond to correct (but 
non-optimal) regenerating codes. Two specific regenerating 
codes are interesting: MSR (Minimum Storage Regenerating 

for a minimum 



codes) offer optimal repair cost 7 
storage cost a = 4^ 



M 



k d-k+1 

and MBR (Minimum Bandwidth 
Regenerating codes) offer optimal storage cost a = ^ 2 d-k+i 
for a minimum repair cost 7 = 77- 2 d-k+i ■ Regenerating 
codes can be implemented using linear codes fl8) , fl9[ be 
they random J24) , f25) (i.e., random linear network codes), or 
deterministic [26 1— [ 30 1 . Similarly to regenerating codes, our 
codes can be implemented using random linear network codes. 

Table [I] gives some examples of storage cost a and repair 
cost 7 for the codes we describe including the coordinated 
codes we propose. These costs depend on the number k of 
devices needed to recover the file, the number d of contacted 
live devices, and the number t of devices being repaired 
simultaneously. Regenerating codes by Dimakis et al. (jTOJ 
represent a clear improvement over erasure correcting codes, 
and our coordinated regenerating codes allow reducing further 



the costs. 



D. Design rationale 



Regenerating codes transfer the minimal quantity of infor- 
mation needed to repair one device storing a bits. Dimakis et 
al. assume fully independent repairs: simultaneous failures are 
fixed independently. The cost of repairs increase linearly in t. 
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Figure 4. Our coordinated regenerating codes encompass all existing codes. 
Moreover, they allow the repair of multiple devices at once without decoding. 

In this work, we investigate coordinately repairing simul- 
taneous failures in an attempt to reduce the cost, along the 
lines of delayed repair in erasure correcting codes. Contrary 
to regenerating codes that repair only using data from live 
devices, our coordinated regenerating codes also allow the use 
of data from other devices being repaired. By coordinating the 
repair, we show that it is possible to repair multiple failures 
at once with an average cost of 7 < 7. As depicted on 
Figure |4j our new codes encompass both erasure correcting 
codes (d — k) and regenerating codes (t — 1). In the 
next section, we detail our coordinated regenerating codes 
supporting coordinated repairs. 

III. Coordinated regenerating codes 

We consider a situation where t devices fail and, repairs 
are performed simultaneously. We assume that an underly- 
ing monitoring service triggers the repair and contacts all 
involved devices (i.e., the t spare devices that join the system 
to replace failed ones). Directly applying erasure correcting 
codes delayed repairs (Fig. lb 1 (i.e., one device repairing for 



many other devices) to regenerating codes (Fig. [TcJ is not 
appropriate. In short, the fact that all the data goes through 
the device that regenerates for others induce more network 
transfers than needed: as repairing does not require decoding, 
gathering all the information at a single device is not necessary. 



Table I 

Some examples of repairs of codes for a file of 32 MB 
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Figure 5. Repairing failures in coordinated regenerating codes. In a network 
of n devices storing a bits, when t devices have failed, t new devices collect 
bits from d live devices. They coordinate by exchanging (t— l)/3' bits with 
other new devices and store a bits. 



A. Repair algorithm 

We introduce new coordinated regenerating codes allowing 
devices to repair simultaneously at the optimal cost (with 
respect to network communication). The repair is illustrated 
on Figures [5] and [7] showing the amounts of information 
transfered, and on Figure [6] showing the computations and 
exchanges of sub-blocks of data. A device being repaired 
performs the three following tasks, jointly with all other 
devices being repaired: 

1. Collect. It downloads a set of sub-blocks (size j3) from 
each of the d live devices. The union of the sets is stored as 

2. Coordinate. It uploads a set of sub-blocks (size /?') to 
each of the t — 1 other devices being repaired. These sets are 
computed from W\. During this step, sub-blocks received 
from the t—1 other devices being repaired are stored as W%. 
The data exchanged during this step can be considered as a 
digest of what each has received during the collecting step. 

3. Store. It stores a set W% of sub-blocks (size a) computed 
from Wi U W2. W\ and W2 can be erased upon completion. 

Interestingly, thanks to the explicit coordination step involving 
all devices, this approach evenly balances the load on all 
devices. Hence, coordinated regenerating codes avoid the 
bottleneck existing in erasure correcting codes delayed repairs 
(i.e., the device gathering all the information) (cf. Fig. lbi. 



In the rest of this section, we present our main results: we 
investigate the optimal tradeoffs between storage and repair 
costs (i.e., optimal values for a (data to store), f3 and j3' (data 
to transfer)). As we consider the problem from an information 
theoretic point of view, we can ignore the nature of the 
information and only consider the amounts of information 
that must be exchanged to repair the redundancy. Our results 
define the fundamental tradeoffs that can be achieved (i.e., 
lower bounds on amounts of information to be transfered to 
repair). 

Our proof is inspired from the proof by Dimakis et al. 
found in |10|. We represent the system as an information 
flow graph. For we add a coordination step, our graph differs 
from the one proposed in fT0[ . However, Lemmas [2] and [3] 
are identical to the ones proposed in flO) as they still apply 
to our information flow graph. When compared to flO) , we 
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Figure 6. Coordinated regenerating codes based on linear codes. The system 
stores a file X and is compound of 5 devices. Device i stores 3 sub-blocks 
{Vi li Hi 3) Vi 3}' Devices 4 and 5 fail and are replaced by devices 6 and 7. 

Table II 

Notation used in SectionIhTI 



k 


Constant (Integer) 


Number of devices needed to recover 


t 


Constant (Integer) 


Number of devices being repaired 


d 


Constant (Integer) 


Number of live devices (d > k) 


a 


Variable (Real) 


Quantity stored 


P 


Variable (Real) 


Quantity transferred (collect) 


P 


Variable (Real) 


Quantity transferred (coordinate) 


7 


Expression (Real) 


Quantity transferred over the network 



allow the coordination of multiple repairs while they assume 
fully independant repairs. 

We determine the optimal codes (i.e., we minimize the 
storage and the repair cost under some constraints obtained 
by studying information flow graphs). We give the expressions 
for optimal values of a (storage at each node), 7 (repair 
cost), f3 (data transferred during collecting step) and /?' (data 
transferred during coordinating step) as a function of d, 
k and t (parameters of the system). We also examine the 
influence of deliberately delaying repairs (i.e., increasing t 
while decreasing d). Our notations are summarized in Table [H] 

B. Information flow graphs 

Our study is based on information flow graphs similar to 
the ones defined in p0) . An information flow graph is a 
representation of a distributed storage system that describes 
how the information about the file stored is communicated 
through the network. The information flow graph Q is a 
directed acyclic graph composed of a source S, intermediary 
nodes x]£, Xc£ OI and x^^t (* corresponds to a time step while 
j corresponds to the index of a device introduced at time step 
i), and data collectors DCi (data collectors try to contact k 
devices to decode and recover the file). The capacities of the 
edges (a, j3, f3') correspond to the amounts of information that 
can be stored or transferred. 

In our approach, a real device x 1 ^ is represented in the 



graph by 3 nodes (x Y 



r ?,3 



and x '4) corresponding to 
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its successive states. The graph of a repair of t devices 
is shown on Figure [7] (assume t divides k.). First, devices 
perform a collecting step represented by edges x a '^ t — > x\£ 
(k < i) of capacity f3 (d such edges). Second, devices undergo 
a coordinating step represented by edges x\£ — > ^coor °f 
capacity /?' for j ^ j' (t — 1 such edges). Devices keep 
everything they obtained during the first step justifying the 
infinite capacities of edges — > x%£ OT . Third, they store a 
using edges x^ OT — > a; out- Figure [i] gives other examples of 
information flow graphs. 

The graph Q evolves as repairs are performed. When a 
repair is performed, a set of nodes is added to the graph 
and the nodes corresponding to failed devices become inactive 
(i.e., subsequently added intermediary nodes or data collectors 
cannot be connected to these nodes). 

The rest of the article relies on the concept of minimum 
cuts in information flow graphs. A cut C between S and DC; 
is a subset of edges of Q such that there is no path from S to 
DC* that does not have at least one edge in C. The minimum 
cut is the cut that has the smallest sum of edge capacities. 

C. Achievable codes 

We define two important properties on codes: 

Correctness A code (n,k,d,t,a,j) is correct iff, for any 
succession of repairs, a data collector can re- 
cover the file by connecting to any k devices. 

Optimality A code (n, k, d, t, a, 7) is optimal iff it is 
correct and any code (n, k, d, t, a, 7) with 
(a, 7) < (a, 7) is not correcj^] 

The following theorem is an important result of our work. 

Theorem 1. A coordinated regenerating code (n, k, d, t, a, 7) 
is correc^ if and only if there exists (3 and j3' such that the 
constraints of ([TJ and (|2]l are satisfied. A code minimizing the 
repair cost 7 ([T|, along constraints of |2} is optimal. 

7 = d/?+(t-l)/3' (1) 

'in this paper, we always consider that (a, b) < (a, b) means that either 
a < a and b < b, or a < a and b < b 

2 We assume that t divides k, no result is known if t does not divide k. 




Figure 7. Information flow graph of a repair of t = 3 devices. The internal 
nodes of the graph represent intermediary steps in the repair. First, each device 
collects f} from d live devices. Second, devices coordinate by exchanging 
0' with each other. Third, they store a. Plain edges correspond to network 
communication and dashed edges correspond to local communication. 



3-1 

Vu, such that Ui = k and 1 < Ui < t, 

i=0 

9-1 i-1 

Ui min{a, (d - ^ Uj)P + (t- Ui)P'} > M (2) 

i=0 j=0 

These constraints, proven in the rest of this subsection, 
mean that the sum of the amounts of information that can 
be downloaded from each of the k devices contacted by 
a data collector must be greater than the file size (amount 
of information needed to recover the original file). To this 
end, we consider, for a given coordinated regenerating code 
(n,k,d,t,a,'f), all (infinite) corresponding information flow 
graphs and evaluate the flow of information that can go from 
the source to any data collector in such graphs. The vector 
u = {ui)o<i< g represents a possible recovery scenario (i.e., 
the number ui of devices contacted in each of the g repair 
group of size t during the recovery). Since the recovery must 
be possible for any scenario, we consider all possible u. We 
show that |2]) is satisfied if and only if decoding is possible at 
any time (i.e., as long as the aforementioned constraints are 
satisfied, no data is lost). 

Lemma 2. For any information flow graph Q, no data 
collector DC can recover the initial file if the minimum cut in 
Q between S and DC is smaller than the initial file size M.. 

Proof: Similarly to the proof in [10], since each edge in 
the information flow graph can be used at most once, and since 
source to data collector capacity is less than the file size A4, 
the recovery of the file is impossible. ■ 

Lemma 3. For any finite information flow graph Q, if the 
minimum of the min-cuts separating the source and each data 
collector is larger than or equal to the file size M, then 
there exists a linear network code such that all data collectors 
can recover the file. Furthermore, randomized network coding 
allows all collectors to recover the file with high probability. 

Proof: Similarly to the proof in | [T0| , since the recon- 
struction problem reduces to multicasting on all possible data 
collectors, the result follows from the results in network coding 
theory which are briefly discussed in Section |H] ■ 

Lemma 4. For any information flow graph Q compounded of 
initial devices that obtain a bits directly from the source S 
and of additional devices that join the graph in groups of t 
devices obtaining j3 from d existing devices and j3' from each 
of the other t—1 joining devices, any data collector DC that 
connects to a subset of k out-nodes of Q satisfies: 

mincut(S', DC) > (3) 
/s-l i-l \ 

min ^2 u i mm 

UE \i=0 J=0 / 

with P = {u : 1 < u % < t A £)f=0 Ul = k }- 

Proof: Let us consider some graph Q (see examples in 
Figure [8]) formed by adding devices according to the repair 
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(c) Q* foru = (2,1,3,...) 
Figure 8. Information flow graphs for which bounds in j5J are matched with equality for some u. 



process described above. Consider a recovery scenario ueP 
in which, a data collector DC connects to a subset of k nodes 
{i^ t : £ /}, where / is the set of contacted devices. 

As all incoming edges of DC have infinite capacity, we only 
examine cuts (U, U) with S G U and {x l ^ t : € 1} C U. 
Moreover some additional cases cannot happen since there is 
an order between x\£, £ cc ? or and x^ t (e.g., x\£ £ U and 
^coor € U needs not be considered). Therefore, we only need 
to examine three cases in the rest of this proof. 

Let C denote the edges in the cut (i.e, the set of edges going 
from U to U). Every directed acyclic graph has a topological 
sorting pT| , which is an ordering of its vertices such that the 
existence of an edge x — >• y implies x < y. In the rest of the 
analysis, we group nodes that were repaired simultaneously. 
Since nodes are sorted, nodes considered at one step cannot 
depend on nodes considered at the following steps. 

First group. Let J be a set of indexes such that {x ^ t : j £ 
Jo} are the topologically first output nodes in U corresponding 
to a first (same) repair. The set contains #{2out : 3 Jo} = 
Mo nodes. Consider a subset Mo C Jo of size m such that 
: j £ M } C U and : j £ J - M } C U. m can 

take any value between and Uq. 

First, consider the m nodes {x { ' J : j £ Mq}. For each node, 
x^f £ U. We consider two cases. 

• If x®g or £ U, then x^J m — > x®'^ t £ C. The contribution 
to the cut is a. 

• If a; coor e U, then x^ x®g OI £ C. The contribution to 
the cut is oo. 



Second, consider the uq — m other nodes {x^ : j € Jo — 
Mo} (third and last case: x\^ , x cc ? or and x l ^ t all belong to U). 
For each node, the contribution comes from multiple sources. 

• The cut contains d edges carrying /3: since xj^. are the 
topologically first output nodes in U, edges come from 
output nodes in U. 

• The cut contains t— uq+tti edges carrying (3' thanks to the 
coordination step. The node x^g OI has t incoming edges 
Xfo —> x®g OI . However, since #({x?; fc } f)U) — u — m, 
the cut contains only t — (uq — m) such edges. 

Therefore, the total contribution of these nodes is 

co(m) > m min(a, oo) + (uq — m)(df3 + (t — uq + m)/3') 

Since the function Co is concave on the interval [0 : uq], the 
contribution can be bounded thanks to Jensen's inequality. 

Co(m) > Mo min{a, d/3 + (t — uq)/3'} 

Second group. Let {x £ t : j £ Ji} be the topologically 
second output nodes in U corresponding to a second (same) 
repair. We follow a similar reasoning. 

First, consider the m nodes {x\^ : j £ Mi} C U. Similarly 
to the above, the contribution of each node is min(a, oo). 

Second, consider the UQ — m nodes {x\^ : j £ J\— Mi} C 
U. For each node, the contribution comes from multiple 
sources. 

• The cut contains at least d — u edges carrying (3: since 
x ^ t are the topologically second output nodes in U, at 
most uq edges come from output nodes in U, and at least 
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d—UQ other edges come from output nodes in U and are 
in the cut. 

• Similarly to the above, the cut contains t — u\+m edges 
carrying (3' thanks to the coordination step. 

Therefore, the total contribution of these nodes is 

ci(m) > ui min{a, (d — Uo)/3 + (t — ui)(3'} 

i-th group. Following the same reasoning, we find that the 
z-th group of nodes (i = 0, ... ,g — 1) in the sorted set U 
contributes 

i-l 

Cj(m) > Ui min{o!, (d — u j)fi + (t ~~ u i)P'} 
j=o 

Summing these contributions for all i, and considering the 
worst case for u € P leads to Q. ■ 
Proof of Theorem U\ From Lemmas [3] and [4] a code is 
correct if it satisfies ([TJ and |2]). 



7 = dp + (t - 1)13' 



(1) 



3-1 



Vu, such that Ui — k and 1 < Ui < t, 



i-i 



^ u. ( min{a, (d - ^ tt 3 -)j9 + (* - > @ 



i=0 



J=0 



From Lemma [2] a code is correct only if mincut(S, DC) > 
Ai. Moreover, for any set of parameter (n, k, d, t, a, (3, (3') and 
any scenario u, we can find a graph Q u such that 

mincut(S, DC) = itj min{a, (d— ^""^ Uj)/3 + (t— Ui)j3'} 

^ (4) 

The graph t/ u is built using the process described here after. 

• The data collector gets all bits from a set U of k devices. 

• The contacted devices repaired simultaneously are 
grouped in subsets Ui of size u, such that U = Uf= ^. 

• Each device x £ U gets /3 bits from all devices in 
{JlZjnUj, f3' from Ui — 1 devices taking part to the 

? 1 

reconstruction, /? from d — X)}=o w j devices not in U, f3' 
from t — Ui devices not taking part to the reconstruction. 

Hence, a code is correct if and only if ([T| and |2]) are 
satisfied. 

A code minimizing (a, 7) under constraints of ([TJ and |2]) 
is optimal as any code with (a, 7) < (a, 7) would not satisfy 
at least one constraint and hence would not be correct. ■ 

D. Optimal tradeoffs 

Determining the optimal tradeoffs boils down to minimizing 
storage cost a and repair cost 7, as defined by ([TJ), under 
constraints of k, d and t are constants and a, f3 and f3' 
are parameters to be optimized. In this subsection, we provide 
optimal tradeoffs (a, 7) between storage cost and repair cost 
(bandwidth). 



1) MBCR codes: Minimum Bandwidth Coordinated Regen- 
erating Codes correspond to optimal codes that provide the 
lowest possible repair cost (bandwidth consumption) 7 while 
minimizing the storage cost a. Figure [10] compares MBCR 
codes to both Dimakis et al. 's MBR [ 10] and erasure correct- 
ing codes with delayed repairs (ECC). Note that similarly to 
MBR, there is no expansion since every bit received is stored 
(i.e., a = 7). 

M 2d + t - 1 



k 2d- k + t 



P = 



M 2 
T 2d- k + t 



M 



1 



k 2d- k + t 



We determine the values for MBCR codes in two step. We 
study two particular cuts and find the minimal values required 
to ensure that the quantity of information going through the 
cuts is at least equal to the file size (thus proving the optimality 
of the solution if it is correct). We then prove that these 
quantities are enough for all possible cuts. 

Proof of MBCR (Optimality): Let us consider two par- 
ticular successions of repairs leading to the graphs shown 
on Figure [8] The repairs corresponding to such graphs are 
described in the Proof of Theorem [TJ As we want to minimize 
7 before a, we assume a > d/3 + (t — l)/3' (i.e., the stored 
amount is always larger than the downloaded amount). 

When Vi, Ui — t (Fig. [Hal, it is required that 



k/t-l 



> M 



which is equivalent to 



M 



k 2d -k + t 



When Vi, Ui = 1 (Fig. 8b 1, it is required that 



i=o 

which is equivalent to 
1 



fe-l / i-l 

^((d-^D/j + Ct-i)/?'] > M 



?> — [ M 3 2d ~ k + 1 
t - 1 I k 2 



the smallest possible 
, the associated repair 



value f3' 
cost is 7 



If we consider 

1 (M _ o 2d-fc+l 
i-l \ k l J 2 

^ + The repair cost hence grows linearly with f3. 

Hence, we should minimize (3. The minimum value for (3 is 

R = M 2 

V k 2d-k+f 

Since lower values would lead to incorrect repair for the 
graphs depicted on Figure [8j these values are optimal. ■ 
Proof of MBCR (Correctness): We have proved that the 
aforementioned values are required for two particular cuts. We 
now prove that such values ensure that enough information 
flow through every cut and, hence, is correct. According to 
Theorem [JJ the following condition is sufficient for the code 
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to be correct. We show that the constraint is satisfied when a, 
(3 and (3' take the values defined for MBCR codes. 

s-i / t-i \ 

Ui min{(d - ^2 u i)P + (* _ u i)/3', a} \ > M 
i=0 \ j=0 J 

since a (the stored part) is always larger than or equal to the 
transmitted data, 

9-1 / i-i \ 

(d - J2 u j)p + (* - ^)/?' > x 

replacing a, /3 and /3' by their values, 

s-i / i-i \ 

^ it, (d- ^ u 3 -)2 + (t - «i) > fc(2d - fc + t) 

i=o \ i=o / 

which is equivalent to, 

9-1 9-1 t— 1 9-1 

9-1 /9-1 \ 2 

(2d + 1) ^2 Ui - I ^ Ui ] > fc(2d - fc + t) 

i=0 \i=0 / 

as k = X)?=o u i> ^ simplifies to 

(2d + t)k-k 2 > k(2d-k + t) 

which is always true. Hence, minimum bandwidth regenerating 
codes are correct. ■ 
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Figure 9. Total repair cost t"f for d = 48 and k = 32. MBCR codes 
permanently outperform both erasure correcting codes and regenerating codes 



2) MSCR codes: Minimum Storage Coordinated Regen- 
erating Codes correspond to optimal codes that provide the 
lowest possible storage cost a while minimizing the repair 
cost 7. This point has been independently characterized by 
Hu et al. in fl3) ; however, they assume that [3 = j3' without 
proving it. We present a simple derivation from Theorem [T] 
allowing to characterize this point. Figure 10 compares MSCR 
codes to both Dimakis et a/.'s MSR 1 1 1 and erasure correcting 
codes with delayed repairs (ECC). Note that for d — k, our 
MSCR codes share the same repair cost as erasure correcting 
codes delayed repair. Yet, in this case, our codes still have 



the advantage that they balance the load evenly thus avoiding 
bottlenecks. 



M 



M 



1 



k d- k+t 



M 



1 



k d- k+t 



Proof of MSCR (Optimality): Let us consider two par- 
ticular successions of repairs leading to the graphs shown 
on Figure [8] The repairs corresponding to such graphs are 
described in the Proof of Theorem Q] 

We minimize a first. It is clear that a — 4r is minimal 
since a < 4^ makes impossible to reconstruct a file of size 
A4 using only k blocks. Hence, what is important is now that 
each element of the sum is at least equal to y, 

2—1 . A 

Vi € . . . g - 1, (d - ^ Ui)P + {t- uW > — 



When Vi, Uj = t (Fig. 8a 1, it is required that 

i-l 

Vie0...k/t-l,{d-^2t)f3> 
3=0 

which is equivalent to 

P> M 1 



M 

k 



k d - k+t 



When Vi, Uj = 1 (Fig. 8b 1, it is required that 

i-l 

Vi e . . . k - 1, (d- + (* - > 

3=0 

which is equivalent to 



M 

k 



^^(^-m-k + D) 

If we consider the smallest possible value j3' = nrzjs ( 



M _ 

(t-l) k 

(3(d—k + 1)), the associated repair cost is 7 = 44 + (fc— l)/3. 
The repair cost hence grows linearly with (3. Hence, we should 
minimize /?. The minimum value for /3 is (3 — 4^ d _\ +t . 

Since lower values would lead to incorrect repair for the 
graphs depicted on Figure [8] these values are optimal. ■ 
Proof of MSCR (Correctness): The proof of correctness 
is quite similar to the previous one. It consists in proving that 

9-1 i-l 

m min{a, (d - V] Uj)(3 + (t - u t )(3'} > M 

i=0 j=0 

is always verified when a, ft and (3' take the aforementioned 
values. 

Since each element of the sum is at most Ui 4^, each element 
of the sum must satisfy the following constraint. 

M t- i M 
Vz < g, min{ — , (d - £ tt,-)j8 + (t - UiJjS'} > 



which simplifies to 



Vi< 9) (d-^tt^ + ft-tii)^ > 
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Applying values for MSCR codes, 



1 i_1 



Vi < g, d — Uj + ti > d — k + 1 

3=0 



which is satisfied if 



Vi < g, Uj < k 



which is true since Ylj=o u i ~ ^ anc ^ u 3 > 0- Therefore, 
MSCR codes are correct. 
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Figure 10. Total repair cost t-y for d = 48 and fc = 32. MSCR codes 
permanently outperform both erasure correcting codes and regenerating codes 



3) General CR codes: The general case corresponds to all 
possible trade-offs in between MSCR and MBCR. Valid points 
(a, /3, j3') can be determined by performing a numerical opti- 



mization of the objective function. Figure 1 1 shows the optimal 



tradeoffs (a, 7): coordinated regenerating codes (t > 1) can go 
beyond the optimal tradeoffs for independent repairs (t = 1) 
defined by regenerating codes by Dimakis et al. [10|. 
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Figure 11. Optimal tradeoffs between storage and repair costs for k = 16 
and d = 24. Regenerating codes (RC) [10] are depicted as t = 1. For each 
t, both MSCR and MBCR are shown. Costs are normalized by M /k. 



E. Optimal threshold 

As previously explained, in regenerating codes, the higher 
the number of devices being contacted d, the higher the savings 



on the repair cost 7. Moreover, when repairs are delayed, 
higher values for the number of devices being repaired t lead to 
higher savings on the repair cost 7. If we consider a system of 
constant size n = d+t, these two objectives are contradictory: 
the longer the delay, the lower the number of live devices 
d. An interesting question is what is the optimal threshold 
t for triggering repairs assuming that d + t is constant (i.e., 
is it useful to deliberately delay repairs?). This question is 
addressed hereafter by studying how MBCR codes and MSCR 
codes behave as t changes in a system of constant size. 

Theorem 5. If we consider a system of size n = d + t, for 
MBCR codes, the optimal value is t = 1 while for MSCR 
codes any value t G {1 . . . n — k} is optimal. 

Proof: Let us consider the repair cost assuming that n = 
d + t is constant. For MBCR codes, the cost 7 = tnZiZ] 
increases when t increases. The optimal value of t for MBCR 
codes is the lowest possible value (i.e., t — 1). For MSCR 
codes, the cost 7 = ^ does not depend on t. The repair 
cost of MSCR remains constant, and t can be set to any 
value as there is no optimum. Neither MSCR nor MBCR 
allow additional gains when deliberately delaying repairs (i.e., 
deliberately setting t > 1). ■ 

Corollary 6. If we consider a system of size n = d + t where 
t can be freely chosen (i.e., the value oft is not constrained 
by the system) both MSR and MBR regenerating codes ^10^ 
are optimal. Hence deliberately delaying repairs to force high 
values for t does not bring additional savings. 

However, if several failures are detected simultaneously, 
coordinated regenerating codes remain more efficient as they 
leverage simultaneous failures by coordinating during repairs. 

IV. Adaptive Regenerating Codes 

In the previous section, we presented coordinated regener- 
ating codes that assume t and d to remain constant across 
repairs. This is similar to regenerating codes p0) , in which d 
remains constant across repairs. Yet, in real systems, it is not 
realistic to assume that the failure rate remains constant over 
time, and to assume that every single failure can be repaired 
before a second one occurs. 

In the particular case of Minimum Storage (a = ^), such 
strong assumptions are not needed. Indeed, when minimizing 
the sum in |2|, we can minimize the different elements of the 
sum, which correspond to repairs, independently. Therefore, 
repairs are independent. We propose to adapt the quantities to 
transfer f3 and f3' to the system state which is defined by the 
number t of devices being repaired and the number d of live 
devices. 

Adaptive regenerating codes simplify the design of a system 
based on regenerating codes. Indeed, when designing such a 
system, at least two parameters must be fixed: the number 
k of devices needed to recover the file and the number d 
of devices needed to repair a file. The higher the d is, the 
lower the repair cost is. Hence, d should be chosen as high as 
possible. Yet, if less than d devices are alive, regenerating 
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codes cannot perform the repair, and the only proven but 
costly way to repair is to decode (gathering the whole file). 
Therefore, d should be fixed to reasonably low values (sub- 
optimal) so that there are always at least d live devices. On the 
contrary, adaptive regenerating codes self-adapt by choosing, 
at each repair, the highest possible d (the lowest repair cost 
7 is achieved for the highest d). Hence, designing a system 
using adaptive regenerating codes implies setting only the right 
value for the parameter k thus leading to simpler designs. 

A. Our approach 

Theorem 7. Adaptive regenerating codes (k, T) are both 
correct and optimal. V is a function (t,d) — > ((3t,di P't d) tnat 
maps a particular repair setting to the amounts of information 
to be transferred during a repair. 



fo,d = 



M 



1 



&t,d = 



M 



1 



(5) 



k d-k + t l ' a k d-k + t 

In this subsection, we prove they are correct and optimal. 

Lemma 8. For any information flow graph Q compounded 
of initial devices that obtain a bits directly from the source 
S and of additional devices that join the graph in groups of 
U devices obtaining j3t i4i from di existing devices and (3' t . d . 
from each of the other ti—l joining devices, any data collector 
DC that connects to a subset of k out-nodes of Q satisfies: 



mm 



mincut(5 ! DC) > (6) 
i-i 

i{a, (di -^Uj)i3 tudi + (U - Ui)P' t . A .} 

3=0 



with P = {u : 1 < u t < ti A J2Lo 



k}. 



Proof: The proof is similar to the proof of Lemma |4] ■ 
Proof of Theorem [7] (Correctness): Using Lemmas [2] [3] 
and [8] we can define the following sufficient condition for the 
code to be correct. The condition is satisfied when /? and f3' 
take the values defined in |5]). 



Vu, such that u- t = k and 1 < Ui < t, 



9-1 



i=0 



^ u t min{^-, (di - ^u,)/3 tiA + (U - Ui)f5' t . d% } > M 



i=0 
i-l 



3=0 



The condition must be satisfied for every u. For any u, since 



M 

» k 



each element of the sum is at most u 
sum must satisfy the following constraint. 

i-l 

k 



each element of the 



M M 
Vi < 3,min{ — , (di - 2_^Uj)P ti ,di + (*i ~ u^Pt^dJ > 



3=0 



k 



which simplifies to 



i— 1 

Vi < g, (di -22uj)0 ti!di + (U - Ui)0' t . 4 . > — 

3=0 



Applying formulas of |5]), 

1 1-1 



3=0 



V« < ,g, di — Uj + ti > di — k + ti 

3=0 

which is satisfied if 

i 

Vi < <7 — 1, Uj < k 

3=0 

which is true since X^=o u 3 = ^ anc ^ u 3 - > ^- Therefore, 
adaptive regenerating codes are correct. ■ 
Proof of Theorem^(Optimality): We prove by contradic- 
tion that the adaptive regenerating codes are optimal. Let us 
assume that there exists a correct code (k, T) such that T < T 
(i.e., for some (t,d), T(t,d) < T(t,d)). This is equivalent to 
(k, F) not being optimal. 

Consider a set of failures such that all repairs are per- 
formed by groups of t devices downloading data from d 
devices. Consider the corresponding information flow graph. 
Assuming repairs are performed with a correct code (k,T), 
the information flow graph also corresponds to a correct code 
(d + t,k,t,d,a,f3 t7d ,f3' td ). 

Moreover, according to the previous section, these fail- 
ures can be repaired optimally using the MSCR code (d + 
t, k, t, d, a, Pt,d, P't <f )• Therefore, there is a contradiction since 
the code (d + t, k,t, d, a, j3t,d, P[ d) cannot be correct if the 
code (d + t,k,t,d,a, f3 t ,d, P't d ) is optimal. A correct code 
(k, T) cannot exist, and the adaptive regenerating code (k, T) 
defined in this section is optimal. ■ 

Building on results from coordinated regenerating codes 
(especially MSCR), we have defined adaptive regenerating 
codes and proved that they are both correct and optimal. These 
codes are of particular interest for dynamic systems where 
failures may occur randomly and simultaneously. 

B. Performance 

We compare our approach to MFR codes defined in fl4) . 
This approach is built upon MSR codes defined by Dimakis 
et al. in |To[ . The coding scheme can be described as (k,V) 
where V is a function d — >• j3 d . The t repairs needed are 
performed independently. 

M 1 



I3 d = 



k d-k+1 



(7) 



Let us consider the particular case where d + t = n. 
The average cost per repair of our codes remains constant 
7 = ^^5^- in the MFR approach, which requires repairs 
to be performed independently, the average repair cost 7' = 

n _"Zl + i increases with t. Therefore, the performance of 
our adaptive regenerating codes does not degrade as the num- 
ber of failures increases, as opposed to the MFR constructed 
upon Dimakis et al. 's codes. This is also shown on Figure 12 
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Figure 12. Average repair cost 7 for n = 64 and fc = 32. Adaptive 
Regenerating Codes (ARC) permanently outperform both erasure correcting 
codes (ECC) and the MFR codes. 



C. Implementation 

Our approach also has significant advantages over the MFR 
approach with respect to the implementation. The imple- 
mentations are similar in principle to the one described in 
Subsection |III-A| and Figure [6] The only difference is that 
the values d and t may differ from one repair to the other. 
Each device stores sub-blocks of data and combines them to 
send the appropriate quantities of information. To be able to 
send B — |^r, each device must store z = 3 sub-blocks. To 
be able to send B = |^ or/3 = j^r, each device must 
store z = 1cm {3, 4} = 12 sub-blocks. Hence, the length of 
any random linear code used to implement such a system is 
I = zk where z is the number of sub-blocks stored by each 
device. We now consider a system of constant size n = d + t 
and compare both implementations. 

The implementation of the MFR approach implies that 
to support d G {k . . . n — 1}, each device must be able 
to send all quantities 8 G {tT'"~^t}' Hence, z = 
lcm {1 ... n - k}. It can be shown that 2 n ~ k < z < 3"" fe . 
Hence, the length of the codes required to implement such 
codes grows exponentially with n — k. 

The implementation of our approach implies that to support 
d G {k . . . n — 1}, each device must be able to send quantities 
8 = -^x^ and 8' = - J - r ^-. Hence, z = n-k. Hence, the 

~ n— k k ' n—k k 

length of the codes required to implement such codes grows 
linearly with n — k. 

Even though simple adaptive regenerating codes (MFR) 
can be built from Dimakis et a/.'s MSR codes, our adaptive 
regenerating codes have two clear advantages. First, their 
repair cost is always lower than any other approach (i.e., 
optimal) and is constant on systems of constant size. Second, 
an optimal implementation of our adaptive regenerating codes 
requires much shorter codes (i.e. smaller I). This is very 
important since I has a direct impact on the complexity of 
all operations (encoding, recoding, and decoding). 

V. Related Work 

As already explained, our coordinated and adaptive re- 
generating codes, which allow optimal simultaneous repairs, 
outperform both regenerating codes [ 10 1 and erasure correcting 



codes with lazy repairs j5], fTT) , |12|. Another approach 
aiming at reducing the repair costs consists in contacting less 
devices while still downloading the same amount from each 
device p2| , [33) . Such approach requires structuring the code 
so that encoded blocks can be recovered without decoding 
by combining a few other encoded blocks. Even though this 
approach offers a reasonably low repair cost, it is neither 
optimal in storage nor optimal in repair cost. Moreover, these 
studies provide storage and repair costs that apply only to the 
specific codes proposed. 

Regenerating codes [ 10] and coordinated regenerating codes 
presented in this paper assume a symmetric role for all devices 
(i.e., they all transfer the same amounts of information). 
Since network connections between every device may not 
be equivalent, it is interesting to adapt the repair strategy to 
take into account the underlying network topology. A first 



study 1 34 1 has focused on structuring the repair as a tree 



instead of a star. Indeed, with regenerating codes, repairs are 
performed by having the repaired device download directly 
from live devices. In [34], a first live device can receive 
information from a second live device so as to forward this 
information to the repaired device. This is of interest in 
systems where devices are connected in a mesh network and 
cannot contact all other devices directly. In such case, this 
can avoid a potential bottleneck link between a live device 
and the repaired device. However, in our study, we assumed 
the most frequent case where all devices are connected to a 
regular network {e.g., a peer-to-peer system) allowing direct 
connection from one device to all other devices. Another 
study [29] has focused on downloading unequal amounts of 
information from other devices during repairs. They define 
the total amount of information that must be downloaded 
depending on the maximum amount of information that can 
be downloaded from each device. They show that the lowest 
repair cost is offered when all devices download the same 
amount of data (i.e., regular regenerating codes). This last 
study can also be applied to our codes and would show 
that allowing unequal downloads (i.e., non symmetric system) 
would also increase the global repair cost. 

Independently from our result, the work |13| addresses 
a subset of the problem we consider. They notice that re- 
generating codes can only repair single failures and come 
up with a solution that can handle multiple failures. They 
naturally define a similar repair method (i.e., they add a 
coordination step to the information flow graph). Yet, their 
solution is much more limited than ours as they only study 
the Minimum Storage case (MSR). Not only, we also study 
the Minimum Bandwidth (MBR) point, but this cannot be 
covered by their model since they assume all transfers are 
equal (i.e., B — 8'). Finally, we also determine numerically 
the general case (i.e., points between Minimum Storage (MSR) 
and Minimum Bandwidth (MBR) points). Their paper is also 
restrictive with respect to system they consider as, they assume 
a system of constant size where all devices are involved (i.e., 
n = d + t). Finally, we do build upon our result to define 
an adaptive form of regenerating codes that is more flexible 
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to use in practical systems while they do not consider such 
constructions. Hence, the previously published paper |13|, 
which is yet another proof of the importance of the considered 
problem, covers only a subset of our results even if it shares 
both the problem and some tools used (an adaptation of 
Information Flow Graphs from Dimakis et al. (9j, |10) ) . 

VI. Conclusion 

In this paper, we have proposed novel and optimal coordi- 
nated regenerating codes by considering simultaneous repairs 
in regenerating codes. This work has both theoretical and 
practical impacts. From a theoretical standpoint, we proved 
that deliberately delaying repairs cannot provide further gain 
in term of communication costs. In practical systems, however, 
where several failures are detected simultaneously, our coordi- 



nated regenerating codes outperform regenerating codes [ 1 1 
and are optimal. We also proposed adaptive regenerating codes 
that allow adapting the repair strategy to the current state of 
the system so that it always performs repairs optimally. For 
both codes, we provided and proved the minimum amounts of 
information that need to be transfered during the repair thus 
defining the fundamental tradeoffs between storage and repair 
costs for coordinately repairing multiple failures. Finally, our 
coordinated regenerating codes can also be viewed as a 
global class of codes that encompass erasure correcting codes 
with delayed repairs (d — k), regenerating codes (t = 1), 
erasure correcting codes (t = 1 and d = k), and new codes 
(t > 1 and d > k) that combine, previously incompatible, 
existing approaches of regenerating codes and delayed repairs 
(cf. Figure BY 

So far, our study has focused on the theoretical framework 
for performing multiple coordinated repairs in distributed 
storage. A straightforward implementation of such codes relies 
on random linear network codes (cf. Figure [6Y Coordinated 
regenerating codes can therefore be used as they are for self- 
healing distributed storage. Yet, the seminal paper on regen- 
erating codes [9 1 has been followed by studies of more con- 
strained repairs (deterministic repairs known as exact repairs 
by opposition with functional repairs studied in this paper). In 
short, in functional repairs, the regenerated redundancy is not 
necessarily the same as the lost one while, in exact repairs, the 
regenerated redundancy is exactly the same as the lost one. A 
recent survey |26| details the various works on exact repairs. 
Since our coordinated regenerating codes only consider the 
problem of functional repair (i.e., they leave open the problem 
of coordinated exact repair), an interesting extension of this 
work would be to study deterministic coordinated regenerating 
codes performing exact repair. Indeed, they would have ap- 
pealing properties (simpler integrity checking, lower decoding 
complexity) when compared to coordinated regenerating codes 
based upon random linear network codes. 
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